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§1 INTRODUCTION 

The purpose of this paper is to repair some inaccuracies in the formulation of the 
main result of [HNl]. As were written, the main theorems 4.1 and 4.2 of [HNl] are 
in fact in contradiction to earlier results of one of the authors [N2]. In the process 
of writing an erratum, we actually discovered some new phenomena. As we found 
these quite interesting, it has lead us to incorporate the needed corrections into this 
self contained article. 

An unfortunate misprint, in which got replaced by R, led the authors to 
misinterpret what their proof in [HNl] actually demonstrated. Upon closer scrutiny, 
we realized that there are two distinct ways to proceed. 

One is that we may still obtain the original conclusions of our main theorems, 
provided we slightly strengthen our original hypothesis (cf. Theorems 5.1 and 
5.2). This however entails a much more complicated argument, involving the CR 
structure of the characteristic bundle, which is of considerable independent interest 
(cf. Theorems 4.3 and 4.6). 

The other is that if we stick to our original hypothesis, then the conclusions we 
obtain are slightly weaker than originally claimed, but in our opinion still interest- 
ing. In fact a new invariant comes into play, which measures the rate of shrinking, 
even in the situation where the local Poincare lemma is valid. 

Recall that here we make an important distinction between the vanishing of the 
cohomology and the validity of the Poincare lemma: Consider the inhomogeneous 
problem Bmu = /, to be solved for w, with given data / satisfying BmI = in some 
domain U containing a point xq. The vanishing of the cohomology in U refers to the 
situation in which, no matter how / is prescribed in U , there is always a solution 
uiivU (i.e., no shrinking). The validity of the Poincare lemma at xq requires only 
that a solution u exist in a smaller domain Vf with xq EVf d U (i.e., there is some 
shrinking which might, in principle, depend on /). Our new invariant measures the 
relative rate of shrinking of Vf with respect to radius(t/), as U shrinks to the point 

Xq. 

Under our original hypothesis, we are able to show that the cohomology of small 
convex neighborhoods of xq is always infinite dimensional, with respect to any 
choice of the Riemannian metric (cf. Theorems 7.2 and 7.3). This means that 
special shapes are needed, if one is to have the vanishing of the cohomology for 
small sets. 
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In §6 we have listed a number of natural examples satisfying the slightly strength- 
ened hypothesis. They illustrate how common it is for the Poincare lemma to fail. 

At the end of §7 we give a very simple example satisfying our original hypothesis. 
It illustrates how, even if the Poincare lemma were to be valid, a shrinking must 
occur which goes like radius(V/) ~ Cr'^/^, where radius([/) ~ r, as r — > 0. 

The first author would like to express his appreciation for the kind hospitality of 
the Universita di Roma "Tor Vergata" and the Humboldt Universitat zu Berlin, and 
in particular Professor Jiirgcn Lcitcrer. The second author would like to express his 
thanks to Fabio Nicola from Turin for pointing out to us inconsistencies in [HNl] . 

§2 A PRIORI ESTIMATES 

In this section we rehearse the facts of functional analysis that we shall use later 
for the discussion of the local cohomology of the ^M-complex. For the proofs we 
refer to [AFN], [Nl] and [HNl]. Let M be a paracompact smooth differentiable 
manifold, of real dimension N. We consider a sequence of complex vector bundles 
{E'^ — > M}q=o,i,... (with E'^ of rank r^), and a complex of linear partial differential 
operators : 

(2.1) C°°(M,£;0) C°°{M,E'-) C'^{M,E'^) > ■■■ 



This means that, for alH = 0, 1, . . . : 

(i) Ai : C°°{M, El) C°°{M, Ei+^) are linear and supp {Aq{u)) C supp(w) for 

aU u e C°°(M, El) (Peetre's theorem), 
(a) Aq^i o Aq = for all q = 0,1, 

We denote by f ^ the sheaf of germs of smooth sections of E'^ and, for every open 
U gM, set £i{U) =C°°{U,Ei). 

For every open U C M and xq G M , we obtain complexes 

{U,£*,A,) s\U) S\U) 8\U) > ••• 

and 

((^0),^%^*) SI 81 81 > ... 

We denote their cohomology groups by : 

ker {Aq:8'i{U) ^ 8'i+^{U)) 



(2.2) H'i{U,8\A,) 



Image (^q_i : 8i-^{U) 8i{U)) 
and 

(2.3) H'i{{xo), 8\ A,) = ^ = % Hi{U, 8*, A,) , 

Image : ' ^ j ^^xo 

respectively. 

When H'i{{xo),8*,A^) = {0}, we say that (2.1) admits the Poincare lemma in 
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By introducing a smooth Riemannian metric in M, and a smooth partition of 
unity {x£, Ug} subordinated to a coordinate triviahzation atlas {U£,X£) for E"^, we 
define for any compact subset K of M the seminorm || • ||q,K,m by: 



(2.4) = E E 



\a\<m 



where fi{x) e is the triviahzation of / in and | • j^,^ is the standard Euchdean 
norm in C'^''. 

Next we introduce smooth Hermitian scalar products ( • | ■ )g on the fibers of the 
E'^'s. This allows us to define the L^-scalar product of /, G £'^{U) by: 



(2.5) / if\9),dX, 

Ju 



where d\ is the measure associated to the Riemannian metric of M. It is well 
defined when supp(/) fl supp((7) is compact in U. Using these scalar products we 
obtain the (formal) adjoint complex 

(2.6) C~(M,E0) C~(M,Ei) C~(M,E2) < ... 

by defining the partial differential operator A* : C°^{M, E^+i) C'^{M, £"?) by: 

^' V/ e C~(M, E^+i), yg e C-^p(M, £;«+!) 

where C^^p{M, E'^^^) is the subspace of e C°°{M, Ei+^) with supp(5f) compact 
in M. 

In [AFN] the following was proved: 

Theorem 2.1. Assume that the complex (2.1) admits the Poincare lemma in 
degree q at the point xq G M, i.e. we assume that the sequence: 

(2-8) ei;' El, Elt' 

is exact. Then, for every open neighborhood uj of xq in M we can find an open 
neighborhood ui of xq in uj such that 

(2.9) 'ife£'^{uj)wii\vAqf = {)inu 3 m e £:''-^(wi) with = / in . 

As a consequence of the open mapping theorem for Frechet spaces we also have : 

Theorem 2.2. Let wi C u; C M be open subsets such that the restriction map 
E*{oj) induces the zero map in cohomology: 

(2.10) {H'^iuj, £*, A,) ^ H^iui, £*, A,)} = , 

i.e. such that (2.9) holds true. Then, for every compact Ki C cui and every integer 
mi > there are a compact K G u>, an integer m > and a constant c > such 
that: the function u in (2.9) can be chosen to satisfy: 

1 1 ^ ^ „ii i-ii 
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Prom the preceding Theorem (see again [AFN]) one obtains: 

Theorem 2.3. Assume that for (2.1) we have (2.9). Then for every compact 
subset Ki of u>i we can find a compact subset K of uj, an integer m > 0, and a 
constant c > such that for every f e ^^^(<^) with Aqf = in uj and for every 
V e £'^{u!i) with supp(v) C Ki we have: 



(2.12) 



/ if\v) 



qdX 



< c\\A*v\\q-i,K^,0 \\f\\q,K,m- 



Using the Hermit ian inner product on the fibers of E'^ and the duahty pairing 
associated to the L^-product, we can define for every open U G M the space 
V"^{U) of i?^-valued distributions in U. We note that to both (2.1) and (2.6) we 
can associate complexes of partial differential operators on distributions, which are 
compatible with the natural inclusion map £'^{U) ^ V'^{U) for every = 0, 1, . . . 
and every open U <Z M. 

We denote by 

ker fAg : D'^(t/) ^ V'+^U) 
(2.13) H''{U,V'*,A,)~ ^ 



Image {Aq_i : V"^-\U) V'\U)) 

and 
(2.14) 

ker (a, : V'l^ ^ 
Image {Ag_i : V% T>'IJ ubxo 

the corresponding cohomology groups. The natural inclusion maps £'^{U) ^ 
V''^{U), induce natural maps in cohomology: 

(2.15) if^(f/,£*, A,) ^ m(U,V'\A,) 
and 

(2.16) H'i{{xo),S\ A,) ^ H'^iix^), V'\A,) 

for every open U C M and xq E M. 
Following the same arguments of [AFN] we can prove the following: 

Theorem 2.4. If (2.16), for fixed q > 1 and xq G M, is the zero map, then for 
every open neighborhood uj of xq in M there exists an open neighborhood uji of xq 
in UJ such that : 

(2.17) Wf e£'^{uj) with Aqf ^0 in UJ 3 u e V"^~^ (ui) with Aq_iu ^ f in ui . 

Moreover, we have the analogue of Theorem 2.3 : 

Theorem 2.5. Assume that for some q>l and open uji C uj C M the composi- 
tion : 



zjar, . c* A \ . zjat. . c* a \ , zjat. . n'* a \ 
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yields the zero map. Then for every compact subset Ki C lji there exist integers 
m, mi > 0, a compact K C uj and a constant c > such that for every f e £'^{00) 
with Aqf = in CO and for every v e £'^{u!i) with supp(v) C Ki we have: 



(2.19) 



{f\v)gdX 



< C\\Alv\\g_l,K^, 



mi 



q,K,m 



§3 Preliminaries on CR manifolds and notation 

In this paper M will be a smooth (C°°) paracompact manifold, of real dimension 
2n + k, with a smooth CR structure of type (n, k): n is its complex CR dimension 
and k its real CR codimension. As an abstract CR manifold M is a triple M = 
(M, HM, J) , where HM is a smooth real vector subbundle of rank 2n of the real 
tangent bundle TM, and where J : HM — > HM is a smooth fiber preserving 
isomorphism such that = — /. It is also required that the formal integrability 
conditions [C°°(M, T^'^M), C°°(M, TO'^M)] c C°°{M,T^'^M) be satisfied. Here 
T^'^M = {X + iJX I X G HM} is the complex subbundle of the complexification 
CHM of HM corresponding to the eigenvalue —i of J; we have T^'^MCiT^'^M = 
and T^^^M®T^'^M = CHM, where T^'^M = T^'^M. When /c = 0, we recover the 
abstract definition of a complex manifold, via the Newlander-Nirenberg theorem. 

We denote by H^M = G T*M \ {X, = VX e H^(^)M} the characteristic 
bundle of M. To each ^ e H^M, we associate the Levi form at ^: 

(3.1) /:^{X) = ^{[JX,X]) = di{X,JX) for XeH^M 

which is Hermitian for the complex structure of H^M defined by J. Here ^ is a 
section of H^M extending ^ and X a section of HM extending X . 

Let S*{M) = 0^!lo'^ £^'^\M) denote the Grassmann algebra of smooth, complex 
valued differential forms on M. We denote by J the ideal of E*{M) that annihilates 



(3.2) Jae0^W(M) 



h>l 



By the formal integrability conditions we have dj C J. We also consider the 
powers of the ideal J, obtaining a decreasing sequence of d-closed ideals of 

S*{M): 

(3.3) S*{M) = J^^J^GJ'^D---Z) J^+k-^ D J^+h D J^+'^+i = {0} . 

Passing to the quotients, the exterior differential defines linear maps: 

Bm ■■ JP/JP+'^ JP/JP+^. The grading of £:*(M) induces a grading of JP/JP+I; 

n 

(3.4) jp/jp+i ^^QP^^^M). 

i=o 

As Bm o Bm = 0, we obtain the tangential Cauchy-Riemann complexes for < p < 
n + k: 

(3.5) (QP'\ Bm) = Io ^ Q^^°(M) ^ Q^'JiM) ^ • • • ^ QlP(M) ^ o\ . 
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We also note that Qm\M) = C'^{M,Q^j^) for complex vector bundles Q^^' on 
M of rank {^^^) + (") and (3.5) is a complex of partial differential operators of 
the first order. We denote by the sheaf of germs of smooth sections / of 
satisfying duf = 0. Note that Qj^ is the trivial complex line bundle over M; we 
set Q% = Om and call it the sheaf of germs of smooth CR functions on M. 

If (M, HM, Jm) and (A^, HN, Jn) are two CR manifolds, we say that a differen- 
tiate map : M ^ iV is Ci? iff: {i) d(p{HM) C HN; (ii) d(f){JMX) = JNd(p{X) 
for all X e HM. 

We say that a CR manifold (M, HM, Jm) of type (n, k) is locally embeddable if 
for each point x G M we can find an open neighborhood U oi x in M, an open 
subset U of C"'"'"'^, and a smooth CR map (j) : U ^ U which is an embedding. 

For a locally embeddable M we shall give now a description in local coordinates 
of the Levi form and of the tangential Cauchy-Riemann complex. 

Let x e M and U open in C"^"'"'^ be as above. We can assume that x is the origin 
of C^+'= and that 

(3.6) M nU = {x e U \ pi{z) = 0, . . . , pk{z) = 0} 
where pi, . . . , pk are real valued smooth functions on U and 

(3.7) dpiiz) A--- Adpkiz) ^0 for zeU. 

The holomorphic tangent space to M at a point z G M, having chosen holomorphic 

coordinates on TgC""*"^, is identified with T^'^M and is described by: 

(3.8) 

dpj{z) 



a=l 



dz^ 



-u"" = Oioi j = l,...,k} . 



We also have: 



(3.9) 



A\...,A^e 



and the Levi form at ^ = (Ylj^i X^d'^pj{z)^ is the complex Hessian 



(3.10) 



k n+fc 



dz<^dzl^ 



u 



"w^ for u = (w") e T]'°M ~ H^M . 



Let z'^'^^ = P + is^ with P , G M, for j = 1, . . . , A;. By a linear change of 
coordinates we can obtain that near a; = 0: 

(3.11) pj{z) = - hj(z\ . . . , t\ . . . ,t'=), with hj = 0(2) for j = 1, . . . , . 
Near x = 0, the complex coordinates z^, . . . ,z'^ and the real coordinates t^, . . . ,t'^ 
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Denote by C = {Ca,j) the matrix 



(3.12) 



C = -2i{l + i 



dhi{z) W ^ f dhj{z) 



dp 



and consider the partial differential operators 
(3.13) 



dz^ ^ ""dP 

On a neighborhood F of in M the natural pull-back composed with the projection 

onto the quotient define an identification: 

(3.14) 



aau-,a^;Pi-l3,dz'^' A • • • A dz'^^ A dz^' A • • • A dz^'' 



l<ai<---<ap<n+A; 
l</3i<---</3<,<n 



where aai,...,ap;f3i...i3g = aai,...,ap;i3i...i3g{z^,---,z'',t^,---,t'') are smooth complex 
valued functions on V. 

With this identification, and observing that [Lq,,L^] = for 1 < a, < n, we 
obtain: 

(3.15) ^ 



d 



M 



l<ai<-"<ap<n+fc 
V i<Pi<---<f3q<n 



A dz'^p A d^^i A • ■ ■ A dzf^-^ 



= Y {L^aa,,...,a^;f3,...i3jdzf^ Adz''' A-'-Adz'^'^ Adzf^' A-'-Adz'^'^ 



l<ai<---<ap<n+k 
l</3i<...</3<,<n 
l</3<n 



§4 The CR structure of the characteristic bundle 

In this section we define regular points of H^M and prove a result that was 
formulated in [Tu, Theorem 2] for a generically embedded CR manifold. In that 
paper we could not find a proof of the stated Levi flatness of the CR structure of 
the conormal bundle. This fact is interesting for our discussion, because it entails 
the regularity of the characteristic codirections ^ G H^M at which the rank of the 
Levi form is maximal. This regularity is requested in Theorems 5.1 and 5.2 to 
discuss the non validity of the Poincare Lemma. 

We have preferred to consider the characteristic bundle, which is more intrinsi- 
cally related to the differential geometry of (abstract) CR manifolds, rather than 
the conormal bundle of [Tu]. However, our results imply those of [Tu, Theo- 
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the characteristic bundle H^M is the image, by the dual map J* of the com- 
plex structure J : TM TM, of the conormal bundle M*{M) of M in M : 

H^M = J* (^M*(M)^, and J* : T*M T*M is biholomorphic. 

Let M be smooth abstract CR submanifold, of type (n, k). Denote by -d the 
tautological 1-form on H^M. If tt : H^M — > M is the natural projection, we have : 

(4.1) ^{X) = ^{tt^X)) if i^HlM and XeT^H^M. 

For each point ^ e H^M, we set : 



(4.2) 



N^H^M = {X e T^H^M I i}{X) = } , 

H^H^M = {X e N^H^M I d'&iX, Y) = VF e T^H^M } . 



Let H^M be the open subset of the nonzero cotangent vectors in H^M. Then 
^ — > N^H^M is a smooth distribution of hyperplanes in TH^M. We note however, 
as the discussion below will clarify, that in general the dimension of H^H^M is not 
constant and therefore ^ H^H^M may fail to be a smooth subbundle of TH^M. 
If U is an open subset of H^M, we define : 



(4.3) 



r m{U) = {X eC°° {U, TH^M) I X(C) e N^H^M G t/ } , 
\ Sj{U) = {X e C°°{U,TH^M)\X{^) e H^H^M G t/}. 



Lemma 4.1 Let U be an open subset of H^M. Then : 

Proof For X e 9){U) and Y e ^{U), we have : 

■d{[X, Y]) = -d-d{X, Y) + X-diY) - Y-diX) = . 

Indeed di}{X,Y) = because X{^) e H^H^M, and i?(r) = 0, i}{X) = because 
y(0, X{^) e N^H^M, for all ^eU. This shows that [i^ft/), <yi(t/)] C %U). 

Let now X,r G f)([/). Then [X, y] G aT([/) and, to complete the proof of the 
Lemma, we need only to verify that d^{[X, Y], Z) = for all Z G C°°(C/, TH^M). 
Since 7^ at each point ^ G f/, we can as well assume that '&{Z) is constant in U . 
We have : 

= dd{X, Z)= Xd{Z) - Zd{X) - d{[X, Z]) 
= -^{[X,Z]) 

and likewise d{\Y, ^]) = 0. Therefore 

^{[X, [F, Z]]) = -d^{X, [F, Z]) + X^([F, Z]) - [F, Z]^(X) = 
and likewise 'd{\Y, [X, Z]]) = 0. Hence we have : 



d^([X, F], Z)= [X, F]z?(Z) - Z^?([X, F]) - ^([[X, F], Z]) 
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This shows that [^:iU), S){U)] C S){U). □ 

Assume now that that we are given a generic CR embedding M M of M into 
an (n + /c)-dimensional complex manifold M. This yields an embedding H^M ^ 
T*M. The cotangent bundle T*M has a complex structure, that we denote by 
5 : TT*M TT*M. We shall still denote by tt : T*M M the natural 
projection, extending H^M M. 

Lemma 4.2 Let x e M and ^ e H^M \ {0}. Then 
(4.5) H^H^M = T^H^M (1 ZT^H^M . 



Proof In the proof we can as well assume that M is an open subset of C"^"'"'^. In 
particular we shall utilize the standard trivialization of the cotangent bundle and 
identify H^M with a submanifold of M x (R2n+2fe^* . 

(4.6) H°M = \^{x,x*) e M X (M2n+2fe)* | {x\H^M) = {0} } . 



We can assume that M is defined in M by 



(4.7) 



M = {xeM\ pi{x) = 0, . . . , pk{x) = 0} , 



with pi, . . . , pkE C'^iM, R) and dpi{x) A • • • A dpk{x) ^ for aU xeM. 
With dp = (d — d)/i, we obtain: 



(4.8) 



k 

x, a-'d'^pj{x) 
i=i 



X e M , a^,...,a^ e 



In this way a tangent vector to H^M at = {^o^ x*^) = \^q^ Si=i '^o'^'^Pji^o) ) is 
identified to a vector in M2n+2fc ^ ^j^2n+2fc^* ^j^^ form: 



(4.9) X=lvo,J2 ^'(d^PjM, ■) + Yl 4dd''Pj{xo){vo, ■)] , 



with vq G Tj^^M and A^, . . . , A'^ G M. We recall the notation J for the complex 
structure of M, and hence here of C"^"'"'^, and ^ for the complex structure of T*M, 
and hence here of T*C"'+'^. We have : 



(4.10) ax = Jvo, ^ A^ (dV,(xo), • ) + 5Z a^o^^(iV,(xo)(^o, J ■ ) 



i=i 



Therefore X and 5^ both belong to T^H°M if, and only if. 
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and there exist real numbers . . .^jjr such that : 



Ej=i ^Hd'^Pjixo), Jw) + aldd'^pj {xo){vo, Jw ) 
^^^"^ = Ej=i I^Hd^Pjixo), w) + Yl^^i alddPpj{xQ){JvQ, w ) 

From this we deduce that : 

a^o {dd''pj{xo)ivo,Jw) - dd''pj{xo){JvQ,w)) 
= P' {d-pj{xo),w) yw e T^,M 



and 



ELi «o {dd''pj(xo){vo,Jw) - dd''pj{xo){Jvo,w)) 



w) 



(4.12) = - Y^l^^ (d^^Pjixo), Jw) = - E ;=i {dpjixo), 

Here the perpendicular is taken with respect to the standard metric of M^""'"^'^. 
We can find the real numbers p^ to satisfy (4.11) if and only if 

k 

J2 4 (dd^Pj (xo) {vo, Jw) - dd^pj {xq) {Jvq, w)) = Mw e H^^M . 

When vq^w & H^qM we have 

dd"" pj {xo){vq, Jw) = -dd''pj{xo){Jvo, w) \/j = l,...,k 
and therefore (4.11) is equivalent to 

(4.13) voekeiC^^. 

Finally we note that (4.12) uniquely determines the real coefficients A-' for any given 

In this way we characterized a vector X e T^^H'^M nZT^oH'^M: it has the form 
(4.9) with 

k k 

(4.14) 2 aidd^Pjivo, Jw) + >^Hdpjixo), w) = ^w E (T^.M)^ . 

i=i i=i 

Next we compute H^^^H^M. To this aim we observe that, denoting by x* the 
dual real variables of (M.'^^-'^'^'^Y we can write a tangent vector in the form: 

(4.15) X= E ^"^ + E^'(^''^^(^)'^«)^ + E«'^^''='i(^)(^'^«)^ 
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where Cq, {a = 1, . . . , 2n + 2fc) is the canonical basis of R2n+2fc ^ £n+k ^^^^ ^^ie 
vector V = Yl'^=i^'^ ^*)'^ belongs to T^M. It is convenient to consider the 
coefficients v'^, to be smooth functions of x, x*, defined everywhere in T*M. A 
vector field of this form belongs to 0T(?7) (for a neighborhood U of in H^M) iff 

k 

(4.16) ^a^{(fpj{x),v) = Q in U. 

i=i 

By differentiation we obtain : 

2n+2fc I f) j ^ I r\ \\ 

(4-17) E Eal^(^'^p.'^) + E«^(^>.w)H^a = o - ^- 

We note that 

k 

(4.18) < = Ea^(d'=p,(x),e,) in C/ 
and therefore 

2n+2fe k r, j 

(4.19) d<= 5] 5]-|;^(dV,-,e«)dx; in C/ . 
Let 

^= E «'°^ + 5ZA*-'((^'Pj(^)'ea)^ + J]a^(^(^V,(2;)(«',e«)^ 

a=l j=l j=l 

be another vector field in U. Let [X, Y] = ES^'' afe + ^«af^) • Then : 

j^a =yP 

OXp OXp 

(4.20) - E/^'(^Vi,e^)|^ 

j=l /3 j=i /3 

Assume that X E ^{U) and impose the condition that d'&{X, Y) = for all 
Y G C°°{U,TH'^M). We can choose F is such a way that ■d{Y) = constant on 
a neighborhood of C/ in T*M, so that : 

oa-' , , , / ow 



(4-21) E {T.£^id^P^M + J2^Ud^p,^^)]dxl = in U 



12 



C.DENSON HILL AND MAURO NACINOVICH 



Then di^lX, Y) = —i}{[X, Y]) and we are lead to the condition : 



(4.22) 



- Ej=i a^dd^pjiv, w') + ^J^i a^dd^pjiw, v') 

~ 5^i=i a^dd''pj{v, w) 

- Yl'i=i a^dd'^pjiv, w') + a^dd''pj{w, v') = , 



where 



k 2n+2k k 2n+2k 

j=l a=l ^-^oc ^-^a 

are the components of v and w orthogonal to HM. The vector X^^ belongs to 
H^^H^M if and only if (4.22) holds at the point = {xo,x*'^) for all choices of 
/Uq, • ■ • 7 Mo ^ ^ every wq G T^^qM. Taking first wq = and arbitrary //q, we 

obtain : 

{d''pj{xo),v'{^o)) ^ for j = 
i.e. v(^o) £ Hj;^M. Next, letting tuo vary in H^gM, we obtain that : 
k 

aidd^pj (v(eo), two) = Vtuo e i?xo^ 

and hence f(^o) ^ ker^^g. Finally, we have: 

k k 
2Y,aidd''pM^o),w'o) + J2K{d'Pjixo),w'o) = V«;(, e T,,M r\ {H,,M)^ . 
i=i j=i 

Since {d^pj.jw) = {dpj,w), and (^(T^,M)^) = T^^.M n{H^,M)^ , by (4.14) the 
proof of the lemma is complete. □ 

Theorem 4.3. Let M be a generic CR submanifold, of type (n, k). Assume that 
the Levi form has constant rank m for aU ^ in an open subset U of H^M. Then 
U is a CR submanifold of T*M, of type {n — m,2k -\- 2m), which is Levi Eat and 
fohated by complex submanifolds of dimension {n — m). Note that the embedding 
U ^ T*M is not generic when m < n. 

Proof We have: (i) [m{U),'yi{U)] C m{U) by Lemma 4.1; (ii) ^(^(t/)) = m{U), 
by Lemma 4.2; (iii) 9^(C/) is a distribution of constant rank (2n — 2m) by Lemma 
4.2 . Hence the real Frobenious theorem provides the foliation, and by the classical 
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We say that a point £ H^M is regular if there exists an open neighborhood 
U of ^0 in H^M and a smooth submanifold V oi U with 

(4.23) ^oeV CU, T^V C H^H^M e F , and T^^V = H^^H^M . 

Since the rank of the Levi form is a lower semicontinuous function of ^ e H^M, 
Theorem 4.3 yields : 

Corollary 4.4 Let M be a generic CR submanifold, of type (n, k), and e 
H^M with rankvC^g = limsupg_>^^ rank£^. Then is regular and there is an open 
neighborhood U of in H^M consisting of regular points ^ with rank>C^ = rankvC^^ . 
We have : 

Lemma 4.5 Let M be a generic CR submanifold, of type (n, k), of a complex 
manifold M. Let £ H^M be regular. If the Levi form C^^ has rank m, then there 
exists an (n — m,)-dimensional smooth complex submanifold V* of a neighborhood 
of Co in T*M, with: 

(4.24) Co e ^* C H^M . 

The projection W = 7r(F*) is a smooth complex submanifold of a neighborhood of 
7r(Co) in M, contained in M. 

Proof The dimension of H^H^M is an upper semicontinuous function of C £ 
H^M. Therefore it remains constant and equal to 2(n — m) on an open neigh- 
borhood of Co in y ■ Let V* be such a neighborhood. We have T^V* = H^H^M 
for aU C e V*. Hence ^T^V* = T^V* for aU i ^V* and then, by the theorem of 
Levi-Civita, V* is a complex submanifold of dimension (n — m) of a neighborhood 
of Co in T*M. Since the fibers of H^M M are totally real, V* is transversal to 
the fibers and therefore the map V* 3 ^ — > 7r(C) G M is a local diffeomorphism. It 
becomes a diffeomorphism after substituting to V* its intersection with a suitable 
small neighborhood of Co in T*M. Finally, 7r(y*) is a complex submanifold of an 
open neighborhood of 7r(Co), contained in M, because the projection T*M M 
is holomorphic. □ 

We have : 

Theorem 4.6. Let M be a generic CR submanifold, of type (n, k), of a complex 
manifold M. Let xq be a point of M and Co G M a regular point of H^M. Then : 
Then there exists an open neighborhood Cj of xq in M , an (n — m)-dimensional 
complex submanifold W of (li with 

Xq eW G M nu) 



and a real valued smooth function p : lu ^ M. with p{x) = for x e M f] Co, 

holomorphic in W for a = 1, . . . , n + k (here z^, . . . , z"^^^ 

W 



d'^p[xo) = Co and 



are holomorphic coordinates in Cj). 
Proof It suffices to consider the situation where M = a) is a neighborhood of xq 

;„ rnn-\-k iti rrt*,-. „ u„ ;j — i-;c — ] j-i ™ ,~. ( rnn-\-k\* i 
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(C"+'=)* is the space of C-linear forms in C""'"'^. If M is described by (4.7), then 
H^M gets identified to the real submanifold of M x (C"+'=)* : 




3 = 1 



z e M , a^,...,a^ e 



By Lemma 4.5, there is a complex submanifold V* of dimension (n — m) of T*M 
that is contained in H^M, contains the point and whose projection W = 7r(y*) 
in M is an (n — m) smooth complex submanifold of M. Define p = 0'''{z)pj{z) 

with real valued smooth functions such that (^z, (l/^) Yl'j=i {^)9Pj{^)^ belongs 
to V* when z eW. □ 



§5 The main theorems (corrected version) 

Theorem 5.1. Let M be a locally embeddable CR manifold of type (n, k). Let 
xq E M and assume that H^^M contains a regular point of H^M such that 
the Levi form has q positive eigenvalues and [n — q) eigenvalues which are 
< 0. Then the local cohomology groups Ti.'^Q {{^o)iQ^)= hnj i?| (U,Q^) are 

UBxo 

inhnite dimensional for all < p < n + k. 

In fact a more general statement is valid. To formulate it, we consider the Bm- 
complex on currents (see [HNl], [NV]) and consider, for an open subset to of M, 
and < p < n + A;, its cohomology groups, that we denote by if| {u>, V QP'*) 
(for < q <n). We define the local cohomology on currents by: 

Note that we have a natural map 

(5.1) Hl^ ((xo), Q^M*) - ((^o)'^' ® 2''*) • 

We have: 

Theorem 5.2. With the same assumptions of Theorem 5.1, the map (5.1) has an 
inhnite dimensional image. 

We give first the proof of Theorem 5.1, then indicate the small changes needed 
to prove Theorem 5.2. 

Proof of Theorem 5.1 

By [BHNl], it suffices to show that ((xq), Q^*) ^ 0. Wc can assume that 

M is a generic CR submanifold of an open subset M of C"^"'"'^, and M is defined by 
(4.7). If -C^o has rank n our statement reduces to [AFN]. If has rank (n — d), we 
utilize Theorem 4.6 . By shrinking M, and changing the holomorphic coordinates, 
we put ourselves in the situation where xq = 0, there is a d-dimensional complex 
linear space W = {z'^'^^ = 0, . . . , 2;"+'^ = 0} such that 
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and 

(5.3) dpi{z) = idz^'^^ in WnM, dpj{0) = idz^'+^lo for j = l,...,/e. 

Set C = {z^, ■ ■ ■ , z"^) and t = (t^, . . . , t^). By the implicit function theorem, after 
shrinking M, we can take the defining functions in the form : 

(5.4) pj{z) = -hj{z), with hj{z) = hj{C,t) = Q{2) , 
and by (5.3) we obtain moreover that 

dh { z^ 

(5.5) hi{z) = ^ and ^\ ' = for zeWr\M and a = d+l,...,n. 
By substituting the complex variable z^'^^ by 

(5.6) z-^^-i V P^z-z^, 
V / dz'^dzf^ 

a,l3=d+l 

and making a linear change of the variables z'^'^^, . . . , 2;"^, we obtain that : 

d+q n 

(5.7) hi{C,t)= J2 ^"^'^ - ^"^" + 0(3) atO. 

a=d+l a=d+q+l 



By Taylor's formula we get : 



z'^ 

dz^dzP 

a,/3=d+l 



a,/3=d+l 

+ I ^ z'^zA . 

\a,l3=d+l J 

By (5.7) the second summand in the right hand side vanishes at 0. Thus we can 
find ro > so that B{ro) — {\z\ < ro} d M and 

h^iz) = h,iCt) < 2j:i+Jd+iZ-z- - (l/2)E:=d+,+i^'^^'^ 

for z e B{ro)nM . 

Let us fix a real number v > 2 and set: 

d+q k 

(5.10) 4>iz) = 4>iC,t) = -ie + h,iC,t)-iy Yl z'^z'^-vY^it^+ih^iCM- 
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Since hj{z) = Qz'^'^^ = inW (1 M, we can find a real number ri with < ri < tq 
such that 

(5.11) 3fJ0(2)<O for zeB{ri)nM. 
Define, for every real r > 

(5.12) fr = el^-^^^)] dz^ A---AdzP A dz'^+^ A • • • A dz'^+'' . 

This is a smooth (p, g)-form, that defines a form in QP'^(i?(ri) fl M) satisfying 

(5.13) dMfT = in 5(ri)nM. 
We next define : 



(5.14) iP{z) ^it^ - hi{z) Yl z^'z'' -uj^it^ +'^hj{C,t)f 

a=l a=d+q+l j=l 



Then we can find a positive r with < r < ri < tq such that 

n-\-k 

2 



(5.15) 



^ n-\-k 

UiIj{z) <--J2^'^^'' for zeB{r)r\M. 



Now v and r are fixed and we set : 

(5.16) Qr = e[^^(^)] dzP+^A---Adz''A---Adz''Adz^A---dz'^Adz'^+'^+^A---Adz''. 

For each r > the form g^- defines an element of Q"+'^-P''^-«(S(r) fl M) and we 
have : 

(5.17) ^M/r = 0, dM9r = in S(r)nM. 

Let X = xiC^i) denote a smooth real valued function defined in M^n+fc g^(,]^ that 
X = 1 for + |tp < i and x = for \(\'^ + |tp > |. If the Poincare lemma is 
valid, we have, for all > sufficiently large, an a priori estimate: 



I 



(5.18) 



< C I sup \D-Dlfr 

zeB{r)nM 
\a\ + \b\<m 



■ {sVip\R{dMX){RC,Rt)/\9r\) 



with constants C = C{R) and m = m{R) which depend on R but are independent 
of T. 

Next we note that 

n k 
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Upon replacing z hj z/ i/r, we have: 

// n k \ 

\ a=l i=l / 

X dz'^ A--- A dz"^^^ Adz^ A-'-Adz"" 

Therefore we obtain that 



(5.19) T-("+t) 



J xiRC Rt)fr 



constant > for r — > 0. 



Next we observe that we have an estimate of the form: 

(5.20) sup \DlD\U\<ciT-'^ 

zeB{r)r\M 
\a\ + \b\<m 

with a positive constant ci which is independent r, because < in B{r) fl M. 
On the other hand we have, for a positive constant C2 : 

(5.21) sup\R{dMX){Rz)/\9r\ < C2-i?-exp(-i?-V(4T)) , 
by (5.15). 

By letting r approach 0, with any fixed large i? > 0, we see that (5.19) cannot 
possibly hold true. This proves the theorem. 

Proof of Theorem 5.2 

The only changes needed to obtain Theorem 5.2 are in formulas (5.18) and (5.21). 
In (5.18) instead of the sup norm of dMgT we need to introduce the sup norm of 
its derivatives up to some finite order mi > 0. This modifies (5.21) by a factor 
^-mi ^YiQ right hand side, but again the right hand side of (5.18) tends to when 
T ^ 0, yielding a contradiction that, in view of Theorem 2.5, proves the statement 
of Theorem 5.2. 



§6 Examples 

1. Let X be the complex manifold consisting of the pairs {Li,Ls) where Lj is a 
complex linear subspace of of dimension i and Li C -L3. This is a compact 
complex manifold of complex dimension 5. Fix the canonical basis 61,62,63,64 of 
C*. A local chart of X in an open neighborhood U of ((61), (61,62,63)) in X is 
given by the coordinates z^, . . . , where Li and L3 are generated, respectively, by 
the first column and by the three columns of the matrix: 




1 



z^ 






1 
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Consider on the structure of a right module over the division ring H of the 
quaternions and let M be the subset of X consisting of the pairs (LijLa) with 
Li • HI C L3. InU the equation of M is: 



1 





1 








1 



—z 
^2 



and therefore it is easy to verify that M is a CR manifold of type (3, 2) and that the 
Levi form of M has in every non-zero characteristic codirection exactly one positive, 
one negative and one zero eigenvalue. Thus at each point of M the Poincare lemma 
fails in degree 1. 

2. Let mi, . . . ,me be integers > 1 and fix complex coordinates w^, w^, zj, . . ., 
C], • • Cp, for j = !,...,£ in c^{mi+-+me)+e+2^ jggj^g ^ subman- 
ifold of type (2[mi + • • • m^] + £, 2) of C^{rm+-+m,)+i+2 ^y. 



M 



j=i h=i 



j=i h=i 



Si 



For every ^ ^ in H'^M, the Levi form has mi + ■ ■ ■ + m£ positive, mi + ■ ■ ■ + m£ 
negative, and £ zero eigenvalues. Thus the Poincare lemma for M is not valid at 
any point of M in dimension q = mi + • — h m£. 

3. Let iV C be defined by 



N 



{z,w) e X 



1 



Z^Z^ + z^z^ 
z^z^ + z^z} 



At each ^ G i/'^M\{0} the Levi form has one positive, one negative and one zero 
eigenvalue. We set = , j = 1,2 and use t^,t^, Z ^ Z ^ z'^ 8iS global coordinates 
on N. The complex vector fields 




a 

d 

a 

a^i 



■ 2 a 

~ '^^ aF 

4 9 

aF 



— tz 



3 a 
aF 



iz 



— iz 



1 a 

at2 



give a basis of T^'^N at each point of A^. According to our resiilt, there exist 
smooth complex valued functions a;i,a;2,'^3, defined in a neighborhood U of in 
R| X such that L^uoj = LjOJu for all 1 < j, /c < 3, but the system LjU = ujj for 
j = 1, 2, 3 has no solution in a neighborhood of 0. We define on M = U xC(^ a CR 
structure by requiring that the vector fields 
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form a basis of T°'^M at each point of M. This M gives an example of a CR 
manifold, of type (4, 2), such that the Levi form has for each ^ e H^M\{0} at least 
3 eigenvalues > and 3 eigenvalues < (actually one positive, one negative and 
two zero eigenvalues), which according to [HI], [H2] is not locally CR embeddable 
near 0. For similar examples see also [HN2] 

4. Let 5"^ be the unit sphere in C'^ and consider the manifold M consisting of the 
complex lines that are tangent to S. This is a CR submanifold, of hypersurface 
type, generically embedded into the complex 4-dimensional manifold M consisting 
of all complex lines of the three dimensional complex projective space CP^. The 
CR dimension of M is 3 and its Levi form has at each nonzero ^ G H^M one 
positive, one negative and one zero eigenvalue. Thus for all xq & M the Poincare 
lemma is not valid at xq in degree 1. 

5. Consider the real hypersurface M in described by : 

M = {^z^ = z^z" + [z" + z^)'^z^z'' } , 

where m is an integer > 2. Then = (^^-^^)(ooo) ^ regular point. Indeed, 
with p = '^zs — z^z^ — {z^ + z^)^z^z^ , we have dp — {i/2)dz^ constant, and hence 
holomorphic, along the holomorphic curve W = {zi = , 2:3 = 0} C M. Therefore 
V = {{z,C)\z e VT, ^ = d^z^/2} C H^M is a holomorphic curve in T*C^. 
Since the Levi form has rank 1, we have in fact T^^V = H^^H^M because 
T^qV C H^^H^M and they have the same dimension. The Levi form has one 
positive and one zero eigenvalue. Then the Poincare lemma fails in degree 1 at 
Xq = (0, 0, 0). Note that in this case the rank of the Levi form is not maximal at 
the regular point and that the rank of the Levi form is not constant in any 
neighborhood of ^0 in H^M. 

6. Let X : M — > M be equal to for t < 1 and equal to t exp(l/(l — t)) when t > 1. 
Then x is smooth and convex non decreasing. Let gi, g2 be positive integers, with 
9i > 2, 52 > I5 and consider the hypersurface in C^^^^^ ; 

M = {{z,w)e X I \z\^ + x{\w\^) = 1 } • 

Then M is the boundary of a smooth bounded convex set and therefore all global 
cohomology groups of the tangential Cauchy-Riemann complex W {M, Q^^) are 
zero for all p = 0, . . . , gi + g2 and 1 < j < gi + g2 — 2 (see e.g. [N2]). However, 
for all points Xq = {zq^Wq) G M with \wq\ < 1, if ^0 ^ -f^So \ {0} the Levi form 
>CgQ has gi — 1 eigenvalues of the same sign and the others equal to zero. Therefore 
the local cohomology groups H'^^~^{{xo), 2m*) infinite dimensional for all p = 
0, ...,gi + g2. 

§7 Further remarks on the Poincare Lemma 

We get back to the general situation considered in §2. Assume that the complex 
(2.1) admits the Poincare lemma in dimension g at some point xq G M. Fix any 
Riemannian metric g on M and denote by B{xq, r) the ball of center xq and radius 
r for the distance defined by g. By Theorem 2.1, for every r > there exists some 
r' > such that (2.9) is valid with u = B{xq, r) and Ui = B{xo, r'). For each r > 



20 



C.DENSON HILL AND MAURO NACINOVICH 



valid in dimension q at xq, and (2.9) does not hold for uj = B{xo, r) and any open 
7^ a;i C a;, we set Kq(r) = 0. We set : 

(7.1) (xo) = limmr and v^lxo) = limsup . 

^ r\o logr ^ r\o ^ogr 

The values i'~ (xq) < [xq) can be cither real numbers > 1 or +oo. [We are making 
the convention that logO = — oo.] If d is the distance associated to the Riemannian 
metric g and di the distance associated to another Riemannian metric gi on M, 
then there exist constants Ci > 0, C2 > 0, Tq > such that : 

(7.2) d{xo,x) < Cidi(a;o,a;) < C2d{xo,x) Va; e M with d{xo,x) < ro . 
Therefore we obtain : 

Lemma 7.1 The numbers iy^{xo) are independent of the Riemannian metric g. 
If there exist a Riemannian metric g and a real tq > such that 

Hi{B{xo, r), £*, A^) = for alio <r< ro, 

then v^ixo) = i^q{xo) = 1. 

The small balls of a Riemannian metric can be considered as convex sets. Thus 
the condition that ^^{xo) > 1 has the meaning that convexity is not sufficient for 
the vanishing of the cohomology, but a small open subset (if there is any) on which 
the cohomology vanishes in degree q needs to have a special shape. 

Suppose that u^{xo) — 1. Then one could say that the cohomology vanishes 
asymptotically at Xq, in dimension q. This occurs for example if Kq{r) ~ cr, with 
< c< 1. 

Suppose that the limit in i'~ (xq) is infinite. Then one could say that the Poincare 
lemma fails asymptotically at Xo, in dimension q. This occurs for example if Kq{r) ~ 
Cexp(— a/r), for positive constants C and a. 

Theorem 7.2. Let M be a locally embeddable CR manifold of type (n, k). Let 
Xq & M and assume that there exists ^ e H^^ such that the Levi form has q 
positive eigenvalues and (n — q) eigenvalues which are < 0. Let g be any Riemannian 
metric on M. Then there are constants ro > and C > such that, for every 
p = 0,1, . . . ,n + k, and < r' < r < tq, the maps : 

(7-3) {B{xo,r), Q^^*) - {B{xo,r'), Q^^*) 

induced by the restriction have infinite dimensional image if r' > Cr^^^. In par- 
ticular (xq) > 3/2. 

In fact a more general statement is valid, considering the ^M-complex on cur- 
rents. We have: 

Theorem 7.3. Let M be a locally embeddable CR manifold of type {n, k). Let 
Xq & M and assume that there exists ^ G H^^^ such that the Levi form has q 
positive eigenvalues and {n — q) eigenvalues which are < 0. Then there are constants 
Tq > and C > such that, for every p — 0,1, . . . ,n + k, and < r' < r < vq, the 
maps : 
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has an inGnite dimeDsional image if r' > Cv^l'^ . 

We give first the proof of Theorem 7.2, then indicate the small changes needed 
to prove Theorem 7.3. 

Proof of Theorem 7.2 

Again by [BHNl], assuming as we can that M is a generic CK submanifold 
of an open subset M of C""*"^, we know that (7.3) has an infinite dimensional 
image whenever it has a non zero image. Thus it will suffice to find conditions on 
< r' < r that are necessary in order that (7.3) has zero image. 

Therefore, we assume that a;o = and M is a closed generic CR submanifold of 
an open neighborhood M of in C""'"'^, where it is given by the equations: 

(7.5) s^=/i^.(C,t) = 0(2) for j = l,...,/c. 

Here 2;"+-' = P + is^ with V ^s^ real for j = 1, . . . , /c and C = (-2^, • • • , z^)- 
We can also assume that 

n n k k 

a=l a=l j=l j,£=^ 

with ea = 1 for 1 < q: < g and ea<Oifg<Q!<n. By substituting the complex 
variable z'^'^^ by z'^'^^ — Y^a=i Si=i ctaj^'^^^'^^ —i S^£=i bj^£z'^~^^ z'^'^^ we can then 
reduce to the case where 

q n 

(7.6) h^{C,t) = Y.z^z^+ ea^"^" + 0(3). 

a=l a=q-\-l 

Let US fix a real number u > 1 + maXq_|_i<Q,<„ \ ea\ and set: 

q k 

(7.7) 0(2) = 0(C, t) = -it' + (C, t) - J] 2"2" - ^ {t^ + ihj (C, t)f 

a=l j=l 



(7.8) fr = ei^"^^^)] dz' A---dzP Adz' A--- A dz'^ 
and 

n k 

(7.9) i;{z) = i;{C,t) = it'-h^{C,t)-iy ^ z^z^ - ly^^it' + ihj{C,t)f 

a=q+l j=l 



(7.10) Qr = e[^^(^)] dzP+' A ■ ■ ■ A d2"+'= A dz''+' A ■ ■ ■ A dz"" . 

Clearly we have 
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We use the same cutoff function x as in §5. If (7.3) has zero image, for all i? > 
with R-^ <r' we have, by Theorem 2.3, an a priori estimate: 



(7.12) 



( 



< c 



\ 



sup \D"fT 

|C|^ + |t|^<r^ 
y |a|<m 



{svip\R{dMx)iRC,Rt)/\gr\) 



with constants C and m which depends on r, r' but are independent of r. 
Next we note that 



(/>(c, t) + V'(c, t) = -^^ E - 2^ E + ^)) ' • 



a=l 



We have: 



x{RC,Rt)frAgr 



= (_l)(n+fe-p)g^n+| f ;^ (i?^(<;, t)) exp ('-.^^'^^"-2^5^(tJ +0(x/^))^^ 



X rf2;^ A • • • A ^2;'^+'= A A • • • A d^'' 



Therefore we obtain that 

(7.13) T-^n+i) 



j x{RC,Rt)fr 



^9r 



-> constant > 



for T — > 0. 

Next we observe that we have an estimate of the form: 



(7.14) sup iD^'frl < exp{cir^/T) 

\a\<m 

with a positive constant ci which is independent of r. Indeed the Taylor series of 
at has a purely imaginary first degree term and a real second degree term which 
is < 0, and the factor involving r""^ is absorbed by the constant ci. 

On the other hand we have, for positive constants 02,03: 

(7.15) sup \R{dMX)iRC, R't)/\9T\ < C3-R- exp {-C2R~'^/t) . 

Indeed the Taylor series of at has a purely imaginary first order term and a 
real second order term which is negative definite, while the form in the left hand 
side is different from zero only for |Cp + |tp > R~^/2. 

Therefore 
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Hence r' < ^/ci/c2 ■ r^/^. This completes the proof of the theorem. 

Proof of Theorem 7.3 

The only changes needed to obtain Theorem 7.3 are in formulas (7.12) and (7.15). 
Both remain valid when instead of the sup norm of BmOt we need to introduce the 
sup norm of its derivatives up to some finite order m\ > 0. In view of Theorem 
2.5, we obtain the statement of Theorem 7.3. 

Example Let us consider, for an integer m > 2, the CR manifold M of type 
(2,1): 

M = {{z\z\ Z^) e I + |^2|2 ^ |^3|2m ^ 

The Levi form is definite and nondegenerate at all points of M where 2;^ 7^ 0. But 
at the points oi S = {\z^\'^ + \z'^\^ = 1, = 0} C M it is degenerate with one 
zero and one non zero eigenvalue. The Poincare lemma in degree 1 is valid at all 
points X eM\S (see [N2] or [AH1],[AH2]), but we have i^i {x) > 3/2 at all points 
X oi S. In particular the cohomology in degree 1 of the intersection of M with a 
small Euclidean ball centered at a; e 5 is always infinite dimensional. And the same 
is true if, instead of small Euclidean balls, one uses small balls in any Euclidean 
metric. Also, like in Example 6, all of the global cohomology groups in degree 1 
are zero, because M is the smooth boundary of a weakly pseudoconvex domain (see 
[BHN2]). 
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